H.264 is a commonly used and widely adopted international video coding or compression standard, also known as Advanced Video Coding (AVC) or Moving Pictures Experts Group (MPEG)-4, Part 10. H.264/AVC significantly improves compression efficiency compared to previous standards, such as H.263+ and MPEG-4. To achieve such a high coding efficiency, H.264 is equipped with a set of tools that enhance prediction of content at the cost of additional computational complexity. In H.264, macro-blocks are used where macro-block (MB) is a term used in video compression, which represents a block of 16 by 16 pixels, though it is noted that different sized N×N blocks could be used as macro-blocks. In the YUV color space model, each macroblock contains 4 8×8 luminance sub-blocks (or Y blocks), 1 U block, and 1 V block (4:2:0, wherein the U and V provide color information). In addition to representation as YUV values, color data can be represented by 4:2:2 or 4:4:4 YCbCr format (Cb and Cr are the blue and red Chrominance components).
Most video systems, such as H.261/3/4 and MPEG-1/2/4, exploit spatial, temporal, and statistical redundancies in the source video. Some macro-blocks belong to more advanced macro-block types, such as skipped and non-skipped macro-blocks. In non-skipped macro-blocks, the encoder determines whether each of 8×8 luminance sub-blocks and 4×4 chrominance sub-blocks of a macro-block is to be encoded, giving the different number of encoded sub-blocks at each macro-block encoding time. It has been found that the correlation of bits between consecutive frames is high. Since the level of redundancy changes from frame to frame, the number of bits per frame is variable, even if the same quantization parameters are used for all frames.
A buffer has been employed to smooth out the variable video output rate and provide a constant video output rate. Rate control has been used to prevent the buffer from over-flowing (resulting in frame skipping) and/or under-flowing (resulting in low channel utilization) in order to achieve acceptable video quality. Motion estimation/motion compensation (ME/MC) plays an important role in most of the video compression standards by efficiently reducing temporal redundancy. One important factor leading to enhancement of coding efficiency in H.264/MPEG4 AVC is the adoption of multiple reference frame motion estimation (MRFME). With MRFME, the encoder is allowed to extend the temporal search range to several previous frames stored in the buffer, hence, a better prediction results. However, the coding gain is achieved at the expense of high complexity, which linearly increases with the number of the reference frames. This restricts its applicability, especially for real-time applications.
Conventional systems for performing MRFME are mainly focused on reducing the complexity for a fixed temporal search range. One conventional way is to decrease the searching points by refining the search pattern or reference frame selection. An algorithm has been proposed to find a precise initial search point from the neighboring and temporal collocated blocks. A scheme to select a portion of the whole reference frames based on sub-sample location and center-biased property has also been suggested. However, any increase in speed with such algorithms has been insignificant. Another algorithm has been proposed based on motion vector concatenation that makes use of motion vector correlation among different reference frames.
Another conventional way for performing MRFME focuses on early termination decision. For example, statistical models have been built to decide whether it is necessary to perform a search on additional frames for each macro-block. The performance gain for MRFME depends highly on the video characteristics in the sense that for some video sequences with certain characteristics, the performance gain can be significant, while for other video sequences with other characteristics, the performance gain can be negligible. Therefore, since computational power for searching is sometimes wasted without any significant benefit, applying a fixed temporal search range is inefficient.
The above-described deficiencies of current designs for H.264/AVC-assisted encoding or compression are merely intended to provide an overview of some of the problems of today's designs, and are not intended to be exhaustive. Other problems with the state of the art and corresponding benefits of the innovation may become further apparent upon review of the following description of various non-limiting embodiments of the innovation.